Repository navigation
Disable static cache on certain MoE models - #39108
ivarflakstad wants to merge 4 commits into
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
run-slow: deepseek_v3 |
|
This comment contains run-slow, running the specified jobs: models: ['models/deepseek_v3'] |
|
|
||
| @auto_docstring | ||
| class DeepseekV3Model(DeepseekV3PreTrainedModel): | ||
| _supports_static_cache = False |
There was a problem hiding this comment.
we only need the one in class DeepseekV3PreTrainedModel(PreTrainedModel) as DeepseekV3Model here is just a subclass of it.
| _tied_weights_keys = ["lm_head.weight"] | ||
| _tp_plan = {"lm_head": "colwise_rep"} | ||
| _pp_plan = {"lm_head": (["hidden_states"], ["logits"])} | ||
| _supports_static_cache = False |
|
|
||
|
|
||
| class DeepseekV3Model(LlamaModel): | ||
| _supports_static_cache = False |
There was a problem hiding this comment.
Wait this isn't the same is it?
There was a problem hiding this comment.
hm, but do we still need this one, if the one in modeling_deepseek_v3 is removed?
There was a problem hiding this comment.
i mean, adding this here means we want it to appear for
class DeepseekV3Model(DeepseekV3PreTrainedModel):
but we don't need it there, so we can remove this one here?
There was a problem hiding this comment.
I'm sure you're right hehe.
Haven't yet had the time to fully understand exactly how modular works 😅
There was a problem hiding this comment.
(me neither just my raw thoughts 😅 )
ydshieh
left a comment
ydshieh
left a comment
There was a problem hiding this comment.
OK, but a nit that is good to change 🙏
Thanks a lot!
|
@ydshieh could you take a look at the results from the model tests I ran? |
|
we should skip the integration test
that one doesn't use the flag to control the test. It's integration test and usually not control by the flag. |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: deepseek_v3, dots1 |
662d1e0 to
ee85f48
Compare
|
run-slow: deepseek_v3, dots1 |
|
This comment contains run-slow, running the specified jobs: models: ['models/deepseek_v3', 'models/dots1'] |
|
run-slow: deepseek_v3, dots1 |
|
This comment contains run-slow, running the specified jobs: models: ['models/deepseek_v3', 'models/dots1'] |
|
Thanks for the PR, and sorry it didn't make it in! This branch is based on v4, and v5 is very different, so rather than rebasing we'd love to invite you to open a new PR against
Closing this one in the meantime, thanks again! |
No description provided.